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Abstract 

Many neurons in the visual cortex are orientation-selective, increase their firing rate 
with contrast and are modulated by attention. What is the cortical circuit that un- 
derlies these computations? We examine how synchrony can be modulated by the 
excitability of interneurons, in a model layer 4 network displaying contrast-invariant 
orientation-tuning. We did not find parameter settings for which the standard ring 
model (Somers et al. (1995)), which contains only cells with simple receptive fields 
(RF), behaved appropriately. Only when interneurons with complex receptive fields 
were included, similar to those found recently in cat primary visual cortex (Hirsch 
et al. (2003)), did the network behave appropriately. A critical feature in the model 
was that complex interneurons projected to simple interneurons but the simple in- 
terneurons did not project back to them. The network was switched from the non- 
attended state to the attended state by increasing the depolarization of the complex 
interneurons. In addition to contrast-invariant orientation tuning, the model repro- 
duced the following experimental results: (1) the gamma- frequency range coherence 
between the estimated local field potential (eLFP) and spike trains of excitatory 
cells was higher in the attended state than in the non-attended state, but the firing 
rates of the excitatory cells did not vary between states; (2) the gamma-frequency- 
range power in the eLFP increased with contrast. The model predicts that there 
are two populations of inhibitory cells, one with complex RF characteristics whose 
firing rate increases with attention and the other with simple RF characteristics 
whose firing rate decreases with attention. 
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1 Introduction 



Inhibitory interneurons make up 15% to 20% of all cortical neurons and have 
been classified into distinct groups based on their morphology, the complement 
of calcium-binding pro teins they express and their physiological properties 



( iMarkram et al.l . 120041 ) . It is commonly thought that interneurons function 



primarily as a brake on recurrent excitation, thereby preventing epilepsy, but 
recent work suggest s that they may also play a direct role in cortical infor 



mation processing (ISanthakumar and Soltesa . |2004 lEchegoven and Soltes2 



2005 



1995 



Llinas et al.l. l2005r) . For instance, in the ring model (IBen-Yishai et al. 



Somers et all Il995l ). orientation-selectivity emerges in part because in- 



hibitory neurons sharpen the weakly orientation-tuned inputs that cortical 
neurons receive from the lateral geniculate nucleus (LGN), whereas recent 
models s uggest that attention may be naediated by the synchrony of inhibitory 



neuro ns (iFries et al.l . l200ll : lBichot et al.l . l2005l : lTiesinga et al.l . l2005l : lBuia and Tiesingal . 



20061 ). It is not clear whether both attention and sharpening of tuning are 



achieved by the same class of interneurons because it has not yet been possi- 
ble to conclusively link the classification of interneurons to their function in 
the cortical circuit. We use computational models to investigate what types of 
interneurons are necessary for attentional modulation of orientation-selective 
neurons in layer 4 of the visual cortex. Our hypothesis is that spatial at- 
tention is mediated by a depo larization of intern e urons , which in turn syn- 
chronize the cortical network (IBuia and Tiesingal . l2006l ). In our simulations 
synchrony could not be modulated effectively in a ring model containing only 
simple inhibitory cells, whereas when the network included complex inhibitory 
cells projecting to simple inhibitory cells, synchrony could be modulated ef- 
fectively. Evidence for complex and simple inhibitory cells was recently ob- 
tained in the visual cortex of the anesthetized cat using intracellular recordings 
( iHirsch et al.l . 120031 ). In addition, anatomical studies have revealed evidence 
for various kinds of comple x-like inhibitory neurons projecting p referentially 
to other inhibitory neurons (IGonchar and Burkhalten . Il999l . |2003| ) and receiv- 
ing top-d own inputs from other cortica l areas or from neurons in the basal 
forebrain (^Freund and Meskenaitel . ll992h . 



The simulations presented here provide predictions for how the firing rates 
of two types of interneurons are differentially modulated with attention. This 
information is not only useful for the in vivo identification of interneuron type 
using multi-electrode recordings, it may also help to correlate their functional 
role with their anatomical and biochemical characteristics using intracellular 
recordings. 
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Table 1 

The parameter settings for synaptic connections in the standard ring model, g is the 
unitary strength of the synapse, Nsyn is the number of synapses, Tgyn is the synapstic 
time constant, 5 is the mean conduction delay, as is the standard deviation of the 
conduction delay 

2 Methods 



2.1 Cortical models 



The ring model consisted of A*";, = 21 columns, each comprised of 21 in- 
hibitory and 84 excitatory cells (total: 1764 excitatory cells, 441 inhibitory 
cells) . These number s are representative for a hypercolumn in cat visual cortex 



( Isomers et al.l . Il995l ). The preferred orientation varied smoothly from column 
to column: for the ith column it was 180°{i — l)/Nc (Figure lA). Since the left- 
most column and the right-most column have similar orientation preferences, 
the model is referred to as a ring model. The LGN was represented using 1681 
ON and 1681 OFF cells laid out on two overlapping 41 by 41 grids spanning 
8 by 8 degrees of the visual field with a center to center spacing of 0.2 degrees 
in both the x as well as the y direction (Figure IB and C). The grids of ON 
and OFF cells were fully overlapping. The LGN inputs to each cortical neuron 
came from 3 rectangular subfields, each 1 by 3 degrees in size, with their long 
axis oriented along the preferred orientation of the neuron (Figure IB and C). 
The OFF, ON and OFF subfields were arranged from left to right with their 
long axes parallel and with a center-to-center distance of 1 degree. For each 
cortical neuron, an appropriate number (see Table 1) of LGN-OFF cells were 
randomly selected out of the area covered by the OFF subfield (Figure IC) 
and were connected to the neuron. A similar procedure was applied to the 
LGN-ON cells, which were selected out of the ON subfield (Figure IB). 

The relative probability P for a connection between a presynaptic neuron i 
(preferred orientation: 9i) and a postsynaptic neuron j (preferred orientation: 9j) 
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Fig. 1. Generation of orientation selectivity in the LGN to cortex projection. (A) 
There were 21 columns with excitatory and inhibitory neurons, whose preferred ori- 
entation smoothly varied from horizontal (0°) via vertical (90°) to almost a 180°. 
The preferred orientation of the rightmost column is close to that of the leftmost 
columns, hence these can be identified as neighbors, giving the model a ring archi- 
tecture. (B,C) There were 1681 LGN-ON and 1681 LGN-OFF cells laid out on a 41 
by 41 grid spanning 8 by 8 visual degrees. An excitatory cortical cell with a receptive 
field center at X = and y = and a preferred orientation of 90°, would receive 
inputs from 12 ON cells located in the rectangle in (B) and inputs from 12 OFF 
cells located in either of the rectangles shown in (C). These rectangles (subfields) 
were rotated and shifted appropriately for a neuron with a different receptive field 
center and preferred orientation. 
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depended on the difference between their preferred orientations, P = f{9i — 
9j,a,mr). Here f{9i — 9j,a,mr) = exp(— a;^/2cr^) for |x| < mr and / = 
for \x\ > mr, a is the orientation tuning width and mr stands for maximum 
range and is the orientation difference beyond which no connections are made. 
For each cortical neuron, an appropriate number of presynaptic neurons were 
chosen with relative probability P and connected to the neuron. The values 
used for the model are summarized in Table 1. 

For the 'complex' version of the model, referred to as the complex ring model, 
11 inhibitory cells with complex RFs were added to each column. These cells 
were labeled by the preferred orientation of the column to which they were 
assigned. They received LGN inputs from ON and OFF cells in a 3 by 3 
degrees area centered on the neuron's receptive field center (Figure 2 B). The 
intracortical connections between complex and simple cells did not depend on 
the difference between the preferred orientation of the pre- and postsynaptic 
neuron (formally: cr = oo and mr = 7r/2). There was an asymmetry in the 
connection between inhibitory neurons with a simple receptive field (SI) and 
those with a complex receptive field (CI). Most inhibitory inputs to CI cells 
came from other CI cells, but a significant fraction of inhibitory inputs to SI 
cells came from CI cells (Table 2). The excitatory cells received inputs from 
both CI and SI cells. 



2.2 Calculation of LGN spike trains used as input to the cortical neurons. 



The LGN cells were represented by non-separable spatiotemporal filters, which 
were defined in terms of a Gaussian spatial filter F{x,y\(7, K) = (i^'/27rcr^) x 
exp(— (a;^+j/^)/2cr^) and an alpha-type temporal filter G'(tlr) = (1/r) exp(— t/r). 
Specifically, F^t = F{x,y\ac,Kc)G{t,Tc)—F{x,y\as,Ks)G{t—6,rs). Here x and 
y are defined with respect to the receptive field center of the LGN cell. Param- 
eters values are Kc/ Kg = 17/16, cxc = 0.17°, as = 0.53°, Tc = 10ms, = 20ms 
and the delay between the surround and center response is 5 = 3ms. The firing 
rate of the ON cells is RoN(t) = [Rq + cunFsdFst * s)]+ and for the OFF cells 
it is RoFpif) = [Rq — Ciin^sd^st * In these expressions Rn = 15Hz is 



the baseline firing rate of thalamic neurons (jSomers et al.l . Il995l ). Qj„ is the 



linear contrast, a scaling factor between and 1 (0 and 100%), s is the stim- 
ulus waveform (see below), Fst * s is the scalar filter output determined as a 
sum across space and a convolution in time, Fgc is a scaling factor such that 
Fsc{Fst * s) is 50 Hz during the sustained part of the response to the vertical 
bar stimulus (see below), and denotes rectification. The spike trains for 
LGN neurons are obtained as a Poisson process with a time- varying rate given 
by Ron it) or RoFpit) for ON and OFF cells, respectively. The filter is 64 ms 
long (sampling rate is 1 kHz), shifting the stimulus onset in the filtered tem- 
poral waveform by 32 ms. Hence, in combination with the 5 to 10 ms axonal 
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Table 2 

The parameter settings for the complex ring model, g is the unitary strength and 
Ntsyn is the number of synapses on the target. The conduction delays and the 
synaptic time constants are as in Table 1. 

conduction delay (see Table 1), stimulus-onset reached cortical neurons after 
a 40 ms delay. 



2.3 Stimulus generation 



The stimulus was represented as a spatio-temporal waveform s{x,y,t), with 
a pixel value of zero representing the gray background. A vertical bar of 1 
by 3 degrees was generated by setting the values of s{x,y,t) equal to one for 
ti < t < t2, -0.5° < X < 0.5° and -1.5° < y < 1.5° during either a 300 ms 
long period starting at ti = 250ms or 400ms, or a 600ms period starting at 
ti = 800ms. We used a temporal resolution of 1 ms and a spatial resolution of 
0.1° for the stimulus matrix. 



2.4 Neuron and synapse models 



The neurons were represented by Hod gkin-Huxley-style n i odels. For the in- 
hibitory neuron, we used the model in (IWang and Buzsakil . 1 19961 ) and for the 
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Fig. 2. The local circuit. (A) The standard ring model consisted of simple excita- 
tory (E) and inhibitory (SI) cells, each with a receptive field comprised of paral- 
lel subfields arranged in an OFF-ON-OFF configuration. The excitatory cells only 
projected to other excitatory and inhibitory cells with a similar orientation pref- 
erence (filled arrows), whereas the inhibitory neurons projected to other neurons 
with orientation preferences in a broader range around that of the home-column 
(open arrows). (B) In the modified network an additional inhibitory cell type (CI) 
with complex receptive field characteristics was introduced. Only the numerically 
dominant projections are shown in the graph. These neurons received inputs from 
LGN-ON and LGN-OFF cells with overlapping receptive fields centers. The CI cells 
projected to the simple excitatory and inhibitory cells, regardless of their preferred 
orientation. A CI cell received excitatory inputs with equal probability from all 
preferred orientations. Only a few SI cells provided inputs to the CI cells. 



excit atory neuron, we used the model in (IGolomb and Amitai 119971 : iGolomb . 



19981 ). The AMPA-typ e excitatory and fast GABA-type inhibit ory synapses 



were also taken from (IGolomb and Amitail . 119971 : iGolombl . Il998l ). The imple 
mentation details and equations for the cell and sy napse model were g i ven in 



a pre vious pubhcation and are not repeated here (iBuia and Tiesingal . 12005 



20061 ) . The only single- neuron parameters varied during the course of the sim- 
ulations reported here are the level of depolarizing current Iqi, I si and Ie 
to the complex inhibitory, simple inhibitory and excitatory neurons, respec- 
tively, and for the excitatory cells, the maximum conductance QksIow (stan- 
dard value 0.075mS/cm^) of slow potassiurn current responsible for adaptation 
f Golomb and Amitai . 19971 : Golombl . 1998 ). and the decay time of the corre- 
sponding kinetic variable ( standard value: = 75ms) . NMDA s ynaps es were 
implemented according to (IGolomb and Amitail . 119971 : iGolombl . Il998l ) with a 
time-scale of 149 ms. We fixed their unitary strength to 32% of the strength 
of the AMPA conductance. 
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2.5 Calculated quantities. 



Spike times were determined as the times the voltage crossed mV from 
below. The firing rate was the number of spikes produced during the stimulus 
period divided by its duration. The rates were averaged across all neurons of a 
given type (excitatory, simple inhibitory, complex inhibitory) in a column. A 
tuning curve was constructed by plotting this firing rate as a function of the 
preferred orientation of the column. We performed the same analysis on the 
synaptic inputs that a cortical neuron receives from LGN neurons and other 
cortical neurons. 



For spectral analysis we us ed the multi taper routines implemented in the 
Chronux MATLAB toolbox dMitra and Pesaranl . [l999l : ljarvis and Mitral . l200lh . 
First, a spike time histogram was constructed for each neuron type by calcu- 
lating the number of spikes in a 1 ms wide bin. Histograms were normalized 
by the number of neurons and the bin width in seconds, yielding the time- 
varying average firing rate expressed in Hz. The power spectrum density of 
the histograms was calculated using the Chronux routine mtspectrumpb with 
a spectral bandwidth NW = 3, averaged over 5 tapers. In this routine, the 
length of the time series was increased to twice the next integer power of two 
by zero-padding. Two types of coherencies were calculated using the routine 
coherencypb with the same parameter values as used for the power spectrum. 
The coherence between histograms of two different types of neurons was cal- 
culated. In addition, the coherence between the histogram of the complex 
inhibitory cells and the spike train of an excitatory neuron was determined 
and averaged across 100 spike trains randomly picked among the 1764 excita- 
tory cells (we excluded spike trains with no spikes, since these led to division 
by zero errors). 



3 Results 



The standard ring model produced an orientation selective response. The 
rastergram and histogram for the excitatory and inhibitory cells are shown in 
Figure 3 A and B, respectively. The mean firing rate of a column in response to 
a vertical bar is shown as a function, fpopuiationiOp) , of its preferred orientation 
9p (Figure 3C, D). Experimentally, an orientation tuning curve, fneuron{(^s), is 
obtained by presenting stimuli of different orientation 6s to a single neuron 
with a specific preferred orientation. Because of the ring symmetry and the 
fact that here the firing rate of a neuron only depends on the absolute value 
of the difference between its preferred orientation and the stimulus orienta- 
tion, these tuning functions are directly related to each other according to 
the mathematical identity fneuron{Os\Op) = f{\0s-0p\) = fpopuiation{Op\Os)- The 
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Fig. 3. Orientation-selective response of the standard ring model to a vertical bar. 
A 1 by 3 degrees vertical bar was presented at full contrast between 400 ms and 
700 ms. (A, B) In each panel, (top) the rastergram and (bottom) the histogram 
are shown for all the (A) excitatory and (B) inhibitory neurons. (C,D) The mean 
firing rate during the stimulus period as a function of the preferred orientation of 
the column, averaged across all (C) excitatory and (D) inhibitory neurons in the 
column. From bottom to top, the tuning curves are for 0, 5, 10, 15, 25, 50, and 100 
% linear contrast. The driving currents (expressed in /zA/cm^) were Ie = —0.6, 
Isi = -0.1. 

tuning function for excitatory neurons was sharper (half width at half height 
(HWHH) was 15°) than that for inhibitory neurons (HWHH=20°). The tuning 
functions were approximately contrast-invariant. Contrast in the simulations 
is a linear scaling factor between zero and one, with zero yielding a baseline 
rate of 15 Hz for all the LGN cells, and one corresponding to the LGN-ON 
cells firing at 65 Hz in response to a 1 by 3 degrees bar covering their entire 
RF. This does not take into account the r ionlin earity of the contrast response 
function of LGN neurons (jCheng et al.l . Il995l ). These nonlinearities can be 
accounted for by an appropriate scaling of the x-coordinate in our graphs. 
However, plotting linear contrast makes it easier to distinguish the nonlinear- 
ity due to cortical dynamics from that due to the LGN input. Furthermore, 
the experimental LGN curves were obtained in response to gratings rather 
than the bars used here and thus may only be qualitatively correct. 



We added the complex inhibitory cells to the network without altering the 
strength of the recurrent excitatory connections. However, in order to obtain 
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Fig. 4. The response of the complex ring model in the non-attended condition. 
A vertical bar was presented at full contrast between 800 and 1400 ms. (A-C) 
We show (top) the rastergram and (bottom) the histogram averaged across (A) 
excitatory cells, (B) simple inhibitory cells and (C) complex inhibitory cells. (D-F) 
The firing rate in response to a vertical bar as a function of the preferred orientation 
of the column for (D) excitatory cells, (E) simple inhibitory cells and (F) complex 
inhibitory cells. From bottom to top, the tuning curves are for 0, 5, 10, 15, 25, 
50, and 100 % linear contrast. The driving currents (expressed in /xA/cm^) were 
Ie = -0.6, 757 = 0.1, Id = 0.4. 

contrast-invariant orientation tuning it was necessary to change the other con- 
nections (compare Tabic 1 with 2). Briefly, the number of inhibitory synapses 
received by each inhibitory neuron was increased from 8 to 40, the total 
strength of recurrent inhibition was increased by a factor four, and the uni- 
tary strength of the excitatory synapse to inhibitory neurons was halved. The 
resulting network activity for the non-attended condition is shown in Figure 
4. There were three obvious changes compared to Figure 3. First, the tun- 
ing was less sharp (Figure 4D, HWHH=17° for excitatory cells and 20.5° for 
inhibitory cells). Second, the contrast sensitivity had decreased (Figure 4D), 
as responses above 10 Hz were only obtained for contrast values of 30% and 
higher, compared with 12% for the standard ring model. Third, the maximum 
rate of inhibitory neurons had decreased from 90 Hz to 50 Hz (Figure 4E) . The 
complex cells were not orientation-selective and their firing rate varied only 
weakly with contrast (Figure 4F). For a 100% contrast stimulus (Figure 4C), 
the complex cells were weakly synchronized with an oscillation frequency of 
about 28 Hz. For lower contrast, the complex cells were not synchronized (see 
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Fig. 5. The response of the complex ring model in the attended condition. A vertical 
bar was presented at full contrast between 800 and 1400 ms. The panels are as in 
Figure 4. The driving currents (expressed in /iA/cm^) were Ie = —0.6, Isi = 0.1, 
Ici = 0.9. 

below and Figure 8). When the CI network is in the non-attended condition, 
an increase in the level of depo larizing current or in the amount of excitatory 
inputs will synchronize it (see (Tiesmga, 20041 )). 



To mimic the effects of attention the depolarizing current to the CI cells was 
increased from Iqi = 0.4 to 0.9 /xA/cm^ (Figure 5). The CI neurons were 
synchronized in the prestimulus period and remained synchronized during 
the stimulus presentation (Figure 5C). In response to the oriented stimulus, 
the excitatory and simple inhibitory cells increased their rate and became 
synchronized. The orientation tuning functions (Figure 5D) were virtually 
identical to those in the non-attended condition (Figure 4D). 

The effect of attention on the contrast response functions was determined for 
neurons whose preferred orientation matched the stimulus orientation (Figure 
6A-C). For contrasts up to 48% (at a firing rate of 36.2 Hz), the firing rate 
of the excitatory cells increased nonlinearly with contrast (Figure 6A). For 
higher contrast, the rate of increase of the firing rate with contrast leveled 
off, yielding approximately 64 Hz at 100% contrast. Note that for even higher 
LGN firing rates, corresponding to contrast values larger than 100%, the firing 
rate still increased with contrast. The CRF in the attended condition is almost 
the same: the difference can be described as a small leftward shift of 5% with 
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Fig. 6. Attentional modulation of the contrast response functions. We show the CRF 
of neurons in the column (A-C) whose preferred orientation matched the stimulus 
orientation and (D-F) whose preferred orientation was orthogonal to the stimulus 
orientation. The CRFs were averaged across (A,D) all excitatory neurons, (B,E) 
simple inhibitory neurons and (C,F) complex inhibitory neurons. The responses are 
shown for a network in the attended (solid lines) and in the non-attended condition 
(dashed lines). 

respect to the curve in the non-attended condition. The CRF of the SI cells was 
reduced in the attended condition compared with the non-attended condition 
(Figure 6B). The firing rate of the CI cells was only weakly contrast dependent 
but was significantly increased with attention (Figure 6C). 

The excitatory firing rate was not altered by attention because of the balance 
between two effects. The increased firing rate of CI cells increases the amount 
of inhibition the excitatory cells receive directly from CIs, but it also decreases 
the firing rate of SI cells, which decreases the amount of SI inhibition to the 
excitatory cells. The overall effect is not straightforward because the degree 
of synchrony of the inhibitory inputs also varied, which by itself, even in the 
absence o f any rate chan g es, co uld modulate the postsynaptic neuron's firing 



rate (see (iTiesinga et al.l . I2OO5I )). The precise balance depends on how much 



of the inhibitory input to the SI cells comes from the CI cells. For the present 
parameter setting, 34% comes from CI cells (Table 2). When the fraction of 
inputs coming from the CI cells was increased, the firing rate of the excitatory 
cells increased significantly with attention. This parameter setting may thus 
be more appropriate for cortical downstream of VI . 

The nonlinearity present in the CRF for low contrast depends on the strength 
of recurrent excitation. For low contrast, the firing rate elicited in response to 
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the stimulus grew over time, because of the recurrent excitation, to reach its 
maximum 100 ms or more after response onset. For this range of contrast val- 
ues, the CRF increased steeply with contrast. For higher contrast, or a higher 
unitary strength of excitatory synapses, the firing rate reached its maximum 
value sooner in time. The leveling-off of the rate of increase of the CRF was 
associated with the firing rate reaching its maximum value shortly after re- 
sponse onset and a change in the dynamics of the interneurons. Specifically, 
in the attended state, the complex interneurons increased their firing rate 
and oscillation frequency but their precision was decreased (see below) . In the 
non-attended state, the simple inhibitory cells increased their firing rate. 

Figure 6D-F shows the CRF for neurons that prefer a horizontal orientation, 
which is orthogonal to the presented stimulus orientation. The excitatory cells 
did not fire for any value of the contrast (Figure 6D), whereas the simple 
inhibitory cells fired but their firing rate decreased as a function of contrast 
(Figure 6E). In the attended condition their CRF was shifted downward by 
approximately 4 Hz compared with the non-attended condition. The complex 
cells labeled as preferring horizontal behaved exactly the same as the ones 
shown in Figure 6C because they were not orientation selective. 

We studied the tuning of synaptic drives from the LGN, SI, CI and E popu- 
lations by calculating for a given stimulus orientation the rate of inputs to a 
neuron as a function of its preferred orientation (Figure 7). Like the orienta- 
tion tuning curves discussed before, these curves can be reinterpreted as the 
input rate to one specific column as a function of stimulus orientation. The ex- 
citatory neurons received moderately tuned input from the LGN (Figure 7D), 
sharply tuned input from other excitatory neurons (Figure 7A, solid line), and 
weakly tuned input from simple inhibitory cells (Figure 7A, dashed line). The 
input from complex inhibitory cells was not orientation-tuned (Figure 7A, dot- 
ted line). The simple inhibitory cells received similarly tuned inputs (Figure 
7B and E), except that the rate of the LGN inputs was lower because fewer 
LGN neurons project to the interneurons (Table 2), whereas the rate of exci- 
tatory inputs was higher. The inhibitory inputs from complex cells dominated 
those coming from the simple inhibitory cells. None of the synaptic inputs to 
complex cells were orientation-tuned (Figure 7C and F). 



We estimated the temporal modulation of synchrony using a multi-taper spec- 
trogram of the spike time histogram of the E, SI and CI populations. The LFP 
is hypothesized to refl ect the synaptic currents to neurons in the neighbor- 
hood of the electrode (ILiu and Newsomd . l2006l ). Hence, when the spike time 
histograms for each population are filtered by the corresponding synaptic time 
constants and added together with the appropriate weights, they might behave 
similar to the LFP. We therefore refer to the (unfiltered) spike time histogram 
as the estimated LFP, eLFP for short. In the non- attended condition (Figure 
8A), complex cells show power in the frequency range between 22 to 28 Hz 
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Fig. 7. Tuning of the synaptic inputs to cortical neurons in response to the presen- 
tation of a vertical bar. We show the rate of inputs to (A) excitatory, (B) simple 
inhibitory and (C) complex inhibitory neurons from excitatory (solid lines), simple 
inhibitory (dashed lines) and complex inhibitory neurons (dotted lines) as a func- 
tion of the neuron's preferred orientation. The responses were averaged across all 
neurons of a specific type in each column. In the bottom three graphs the LGN 
inputs to (D) excitatory, (E) simple inhibitory and (F) complex inhibitory neurons 
are shown. 



which grows in amplitude and frequency as a function of contrast. The corre- 
sponding oscillations are only visible in the spike time histogram for the 100% 
contrast stimulus (Figure 4C). Power at this frequency was absent or barely 
detectable in the simple inhibitory and excitatory cells (Figure 8B). In the 
attended state (Figure 8C and D), the complex cells were synchronized dur- 
ing the entire period, including when there was no stimulus present. During 
stimulus presentation the synchrony was maintained, but for higher contrast 
the oscillation frequency increased whereas its power decreased slightly (Fig- 
ure 8C). The simple inhibitory and excitatory cells followed the CI generated 
rhythm when they spiked, that is, during the stimulus period. The gamma- 
frequency range power in the eLFP thus increased with contrast (Figure 8D). 

The coherence between the excitato ry and inhibitory population was quanti- 
fied u sing the multi-taper coherency (IMitra and Pesararu . ll999l : lJarvis and Mitral . 



20011 ). We calculated the coherencies between the spike time histograms (not 
shown) as well as between the excitatory spike trains and the eLFP of the CI 
cells (Figure 9). The latter measure is similar to the spike field coherence used 
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Fig. 8. Gamma- frequency-range power in the estimated local field potential in- 
creased with contrast. We show the power spectrum density of the spike time 
histogram (eLFP) of the (A,C) complex inhibitory neurons and (B,D) excitatory 
neurons in the (A,B) non-attended state and (C,D) attended state. In each panel, 
power spectra are shown for different values of the contrast: (A), from bottom to 
top, 0, 35 and 100%; (B), from bottom to top, 0, 25, 50 and 100%; (C), left 0% 
and right 100%; (D) from bottom to top, 0, 10, 25 and 100%. The results are for 
a complex ring model in the attended state (as in Figure 5). The power spectrum 
was calculated across 600 samples of the spike time histogram with a 1 ms time 
resolution, and averaged across 5 tapers with a bandwidth of NW=3. 



in (iFries et al.l . l200ll ). The coherence is a complex quantity - its absolute value 
represents the strength of the coherence (Figure 9A and C) and its phase is 
proportional to the delay between the activity of the two populations (Figure 
9B and D). During the stimulus period in the attended state, the coherence 
showed a clear peak for frequencies between 28 and 40 Hz (Figure 9A) with 
the CIs lagging the excitatory neurons, as is indicated by the negative phase 
(Figure 9B). The coherence grew with contrast, leveling off at 70% contrast 
(Figure 9C). Likewise, the phase lag increased, becoming more negative with 
contrast (Figure 9D). In the non-attended state, the level of coherence was 
not significant when tested at a p-value of 0.05. However, in the 22 to 32 Hz 
frequency range, there was coherence between the eLFPs of the excitatory and 
CI neurons (not shown). 

Is it possible to obtain synchrony modulation without adding complex cells 
to the network? Theoretical and computational studies have identified two 
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Fig. 9. The gamma-frequency-range coherence between the estimated local field 
potential and the excitatory neurons increased with contrast. The (A) coherence 
and (B) relative phase between the spike time histogram of inhibitory neurons and 
the spike trains of excitatory neurons as a function of frequency. A negative phase 
means that the excitatory neurons are ahead of the inhibitory neurons. The (C) 
coherence and (D) relative phase in the 28 to 36 Hz frequency band is plotted as 
a function of stimulus contrast. The results are for a complex ring model in the 
attended state (as in Figure 5) and are averaged across a hundred spike trains of 
excitatory neurons. The coherence was calculated across 600 samples of the spike 
time histogram with a 1 ms time resolution, and averaged across 5 tapers with a 
bandwidth of NW=3. 



range 


(Wane and Buzsaki 


. 1996 


: White et al 




1998: InI. 2000: Tiesinera 


et al.. 


2nni: . 


4.radi and Solteszl. ^ 


2OO2I; I 


3artos et al.. 20021: 


Boreers and KoDeU 


2OO3I: 


Brunei and Wand. 20031: 


Hansel and Matd. ^ 


>003|: 


Boreers and Kopeh. 


2OO5I: 


Vidaet all. 2006). In the first one. "PING" 


Boreers and Kopeli 2003f). a 



synchronous volley of spikes from excitatory cells elicits a volley of spikes 
from the inhibitory cells, which shuts down the network for approximately 
a gamma period, 25 ms, after which the cycle starts anew. For this rhythm 
to be stable, the inhibitory cells s hould not be able to spike before the syn- 
chronous excitatory volley arrives (IBorgers and Kopelll . l2005l ). In the network 
studied here, excitatory and inhibitory cells are both driven by almost si- 
multaneous LGN inputs that can make them spike even without intracorti- 
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cal excitation, suggesting that the PING mechanism might not be effective 
in this network architecture. In the second one, "ING", the inhibitory net - 



work synchronizes by way of mutual inhibition (jWang and Buzsakil . 119961 ) . 
This rhythm r equires about one hundred neurons and a high degre e of inter- 
connectedness ( iWang and Buzsakil . ll996l : lGolomb and Hanseilioooh . Further- 
more, it can only exist when the random background activity (noise) and the 
degree of heterogeneity (d i fferences in intrinsic excitability) are s r aall e nough 
flWang and Buzsakil . Il996l : Iwhite et al.l Il998l : IXiesinga and Josel . l2000h . It is 
the heterogeneity that posed the main problem in our explorations of the 
standard ring model. In Figure 7B and D the rates of intracortical and LGN 
inputs to interneurons are shown. The interneurons whose preferred orienta- 
tion matched the stimulus orientation received much stronger inputs from the 
excitatory neurons and LGN neurons than did those whose preferred orienta- 
tion was different from the stimulus orientation. The intracortical connections 
contributed more to the heterogeneity than the LGN inputs. We explored how 
far we could get, using only simple cells, towards our goal of finding a network 
that is synchronous in the prestimulus period and remains synchronous after 
the onset of the stimulus. The number of inhibitory cells was tripled to 63 
per column and the number of inhibitory synapses onto each interneuron was 
increased to 128. Since the intracortical inputs caused the most heterogeneity, 
the intracortical connections were either made independent of the difference 
in orientation preference between the presynaptic cell and the postsynaptic 
cell or their unitary strength was reduced. In addition, the unitary strength 
of the thalamocortical synapses was reduced slightly. We found three ways in 
which the increase in heterogeneity associated with stimulus onset could be 
absorbed by the network without losing synchrony. First, the network could 
increase its oscillation frequency (Figure lOA). Second, it could decrease the 
latency of the interneurons whose preferred orientation matched that of the 
stimulus without changing the firing rate (Figure lOB). Third, it could in- 
crease the number of spikes the interneuron produces on each cycle without 
changing the oscillation (Figure IOC). The parameter settings are summarized 
in Table 3. 



In case 1 (Figure lOA), the interneurons received a large depolarizing drive 
and the mutual inhibition was strong. As a result, the interneurons could syn- 
chronize without strong LGN inputs. The intracortical connections were made 
untuned, that is, the connection probability did not depend on the difference 
in preferred orientation between the presynaptic neuron and the postsynaptic 
neuron. When the vertical bar came on, it increased the rate of the interneu- 
rons with a preferred orientation close to 90°. These neurons led the oscillation, 
thus increasing its frequency. Because of the strong mutual inhibition the in- 
terneurons with preferred orientations farther from 90° remained entrained to 
the sped-up rhythm, but they occasionally skipped a cycle. The modulation 
of interneuron firing rate with preferred orientation was weak: the dynamic 
range - the difference between the highest and lowest firing rate - was about 
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Fig. 10. Alternative mechanisms for obtaining synchrony modulations in the stan- 
dard ring model. In each panel, we show (top) the rastergram, (middle) the his- 
togram of simple inhibitory cells and (bottom) the firing rate in response to a 
vertical bar as a function of the preferred orientation for excitatory (solid lines) and 
simple inhibitory cells (dashed lines). We present simulations for three cases, when 
stimulus presentation increased the oscillation frequency (A), reduced the latency 
(B), or increased the number of spikes per cycle (C) of neurons with a preferred 
orientation that matched the stimulus orientation. The parameter values are listed 
in Table 3. 

15 Hz. This modulation was on top of a high basehne rate of about 46 Hz. 
The dynamic range of the excitatory neurons was similar, around 18 Hz, with 
a baseline rate of 2.5 Hz. 

For case 2 (Figure lOB), the interneurons still received a large depolarizing 
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Parameters for the simulations shown in Figure 10. The parameters are as in the 
standard ring model, except that there were 63 inhibitory cells (increased from 
21) and each inhibitory cell received 128 inhibitory synapses. The thalamocorti- 
cal synapses onto excitatory cells had a unitary strength of 12//S/cm^, those on 
inhibitory cells were 7/xS/cm^. Ie and // are the currents to the excitatory and 
inhibitory neurons, respectively. 

current, but now the excitatory neurons also received a depolarizing current. 
The intracortical connections were made orientation-selective again, but the 
strength of all intracortical connections was reduced except for the strength 
of the mutual inhibition, which was kept the same. For these parameters, the 
excitatory neurons fired at a high rate even before stimulus onset and the 
network was synchronized. When the vertical bar came on, it increased the 
firing rate of the excitatory neurons with an orientation preference close to 90°. 
However, because the strength of the excitatory synapses on the interneurons 
was weak, the stimulus onset did not increase interneuron firing rate, rather it 
decreased the spiking latency of interneurons preferring vertical bars compared 
with those that preferred different orientations. The inhibitory cells were not 
orientation-selective in the classical sense since their firing rate did not change 
with stimulus orientation. The dynamic range of the excitatory neuron was 
similar to that in case 1, about 24 Hz, but it sat on top of a much higher 
baseline rate of 35 Hz. 

For case 3 (Figure IOC), the depolarizing current to excitatory and inhibitory 
neurons was reduced. The unitary strength of the intracortical connections was 
returned to their values for case 1, but they remained orientation-selective as 
in case 2. Before stimulus onset, the excitatory and inhibitory neurons were 
synchronized with an oscillation frequency of about 25 Hz. The excitatory 
neurons fired one spike on each cycle and led the interneurons, which fired 
two spikes on each cycle. When the vertical bar stimulus came on, the in- 
terneurons with an orientation preference close to 90° increased their firing 
rate and spiked three times on each cycle, whereas the interneurons preferring 
orthogonal orientations reduced their rate to about one spike per cycle. The 
oscillation frequency increased slightly to 28.6 Hz. The excitatory neurons pre- 
ferring vertical stimuli also increased their firing rate, but they still spiked at 
most once on each oscillation cycle. The dynamic range of the interneurons 
was large, 66 Hz, on top of a baseline rate of about 31 Hz, whereas the exci- 
tatory neurons had a small dynamic range of about 19 Hz on top of a 15 Hz 
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baseline. 



The outcome of this exploration is that it is possible to modulate synchrony. 
However it is not possible, at the same time, to significantly sharpen the 
orientation tuning of cortical neurons compared with that of the LGN inputs 
by way of recurrent excitation and to have a large dynamic range of the firing 
rate. 



4 Discussion 



Multiple features of a visual stimulus are represented in the neural activity 
which it elicits in the visual pathway. These features include orientation, lu- 
minance contrast and whether the stimulus is in the focus of spatial attention. 
The most frequently used measure of neural activity is firing rate. Consider 
an orientation-selective neuron that has a baseline rate of 5 Hz, responds to a 
stimulus that matches its preferred orientation with 30 Hz if the stimulus is at 
50% contrast and with 60 Hz if it is at 100% contrast. If a rate of 25 Hz is mea- 
sured, what could the stimulus have been? It could have been a high contrast 
stimulus with an orientation different from the neuron's preferred orientation, 
or a low contrast stimulus with the neuron's preferred orientation, or a low con- 
trast stimulus that was in the focus of attention. Or it could be that no stim- 
ulus was presented at all, but that the neuron's baseline firing rate increased 
because of general arousal. How does the visual system disambiguate the dif- 
ferent possible meanings of such a firing rate response? And, are the changes in 
activity due to stimulus identity, stimulus strength and the focus of attention 
generated by distinct circuitry? The answer to the first question is that the 
population activity must be used, with the identity of the neurons that are 
most strongly activated and the degree of coherence/correlation between them 
being the most significant aspect of the population activity. So far, electro- 
physiological recordings in non-human primates have only offered a glimpse of 
the richness of the population response. Nevertheless it has become clear that 
in the primary visual cortex of anesthetized macaque mo nkeys the LFP power 



in th e gamma frequency range increases with contrast (iHenrie and Shapley 



20051 ) , whereas the gamma- frequency-band co herence between a neuron's spike 



train and the LFP increases with attention ( Fries et al.l . l200ll : iBichot et al. 



20051 ). Thus, stimulus strength and the focus of attention modulate neural 



correlations (jSalinas and Sejnowskil . l200ll ). There is no definite answer to the 



second question, primarily because there is a lack of models and experiments 
that have addressed the issue of how attention, contrast and stimulus identity 
interact. This study is our first attempt to resolve this issue. 

Our goal was to determine how the effects of attention could be incorporated in 
a model that produces contrast-invariant orientation-selective responses. The 
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model needed to reproduce the following observations. First, contrast-invariant 
orientati on tuning c urves should have a half width at half height between 10° 
and 30° (iGur et all 120051 ) . Second, the power of the local field potential (LFP) 
in th e gamma-frequency should increase with contrast (IHenrie and Shapleyl . 
20051 ) ■ Third, the w idth of the orientation curve sh ould not change with at- 
tentional condition (IMcAdams and Maunselll . Il999l ). Fourth, attention should 
induce on ly small changes in th e firing rate of neurons in responding to a simple 
stimulus (IMcAdams CJl . l2005l ). Fifth, attention should modulate the power of 
the LFP, as well as the coherence of spike trains with the LFP, in the ga mma 



frequency range (IFries et al.l . l200ll : iBichot et all l2005l : iTaylor et al.l . l2005l ) . We 



show that it is easiest to assume the existence of interneurons with complex re- 
ceptive field characteristics, that are weakly or not at all orientation-selective 
and that show a modest increase in firing rate with contrast, but that are 
strongly modulated by attentional state. We have not conclusively shown that 
similar dynamics could not be achieved using only excitatory and inhibitory 
cells with simple receptive field characteristics, but so far we have not been 
successful in finding such a network that behaves appropriately. The reason for 
this is that the inhibitory neurons had to serve two functions. They needed to 
sharpen orientation-selectivity and they needed to synchronize. Synchrony de- 
velops in interneuron networks when (1) there are enough interneurons, about 
one hundred (Wang and B uzsaki, 1996), (2) they are well connected, about 
sixty synapses per neuron (jWang and Buzsakil. Il996r). f3 ) the level of back- 
ground activity, noise, is low enough ( Tiesinga and Jose . 2000l) and (4) the 



degre e of heterogeneity is low enough (jWang and Buzsakil . Il996l : IWhite et al. 



19981 ). Not only does the variability in intrinsic properties of neurons con- 
tribute to the heterogeneity, but the inhomogeneous activation of neurons also 
contributes because of their different preferred orientations. Our solution was 
to assign a different interneuron to each function. In experiments, two types 
of interneurons were found in layer 4 of c at pr imary visual cortex in approx- 



imately equal proportions (iHirsch et al.l . l2003l ). One had a simple receptive 



field and was orientation-selective, whereas the other had a complex receptive 
field and was not orientation-selective. In a recent model these complex in- 
terneurons were show' n to b e useful for contrast-invariant orientation tuning 
(ILauritzen and Millen . |2003| ). The question remains whether these complex 
interneurons also exist in layer 4 of primate visual cortex. The mechanism 
proposed here predicts that there is a pool of interneurons with complex RF 
characteristics that increases their rate with attention and another pool of 
interneurons with simple RFs that decreases their rate with attention. 



We studied the behavior of the model in two operating regimes. In the first, the 
activity before stimulus onset was asynchronous, but the onset of a sufficiently 
high contrast stimulus led to weak synchronous oscillations in the spike time 
histogram of complex inhibitory cells. Hence, the gamma-frequency power in 
the eLFP associated with the CI cells increased with stim ulus contrast, consis- 
tent with the previously mentioned experimental results (IHenrie and Shapleyl . 
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20051 ). We propose that this parameter setting represents the non-attended 
state. In the second, the complex cell activity was synchronous before and 
also during stimulus presentation, but the oscillation frequency increased dur- 
ing the stimulus presentation when the contrast was high enough. The ex- 
citatory cells were also synchronized. Their rate was low for low contrast, 
but it increased with contrast. As a result, the coherence between excitatory 
activity and the eLFP (from CI cells) increased with contrast. We propose 
that this parameter setting represents the attended state. The network can 
be switched from the non-attended to attended state solely by increasing the 

his d rive could be supplied by various 



subcortical ( 


Freund and Meskenaite, 


1992 


) and cortical feedback projections 


(Gonchar and Burkhaltei 




1999. 


2003 


)• 



The preceding discussion has focused on the modulation of synchrony by at- 



tention have been reported ( 


Luck et al. 




1997; 


McAdams and Maunsell, 


1999; 


Treue and Maunsell 


1999; 


Revnolds et al.. 


2000; 


Williford and Maunsell 


2006) 



In VI, no clear evidence for f iring rate chariges wi th attention have been found 
when using simple s timul i ( McAdams CJ . 20051 ) (but see ( Roelfsema et al. 



2004 : iKhayat et al.l . l2006l )). However, attention may cause chang es in syn- 
chron y in VI, which could lead to firing rate increases downstream (ITiesingal . 
20041 ). Hence, our parameter settings for the VI model were chosen such that 
there was little change in the CRF of excitatory neurons whose preferred ori- 
entation matched the stimulus orientation. This was achieved by tuning the 
number of synapses from CI to SI cells such that the increase in inhibition from 
CI cells to the excitatory cells with attention was balanced by the decrease in 
inhibition from SI cells. An increase in excitatory activity with attention, as 
is observed in V4, can be obtained in this type of network by decreasing the 
fraction of synapses on SI neurons coming from CI cells. 



We used the ring model to genera te orientation selective neurons. T h e ring 



model has experimental support (ITsodyks et al.l . Il999l ; iKenet et al.l. 12003 ) 



and has been used previously in large-scale model simulations (jSomers et al. 



19951 ). There are alternative models, some of which require strong recur - 
rent excitation and inhibition (iMcLaughlin et al.l . l2000l : iMarino et al.l . |2005| ). 
others of wh i ch are based primarily on the selectivity of thalamic inputs 



( iTroyer et al.l . Il998l ; iFerster and Millerl. |2000|). The actual situation niay li e 



between these two extreme cases ( iMonier et al.l . |2003| ; iTeich and Qianl . |2006| ) . 
We have chosen the ring model as the first case to study, but it is of interest 
to determine how the considerations presented here apply to the alternative 
models. 



Experiments with multi-electrode arrays in awake behaving primates are presently 
being conductedin a number of labs, making it possible to record from multiple 
neurons simultaneously during an attention-demanding task. The model can 
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potentially be useful for these experiments since it predicts how the coherence 
between neurons changes with attention and contrast and how it depends on 
the difference in stimulus preference between the neurons. 
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